Influenza Trend Prediction Using Kalman Filter and Particle Filter

Background. Seasonal influenza can cause severe health problems and significant economic burdens in various regions of the world. In addition to the substantial morbidity and mortality cases caused by influenza, the emergency department crowding is also partially attributed to the influenza patients. Forecasting the influenza trends is crucial in developing effective countermeasures to mitigate an epidemic outbreak and allows medical facilities to allocate resources accordingly. Aim. Filtering techniques are studied to model various dynamic systems, because they provide better estimations through recursive Bayesian updates. We review and implement several filtering techniques to predict the influenza trend in short term. Data. We studied both the synthetic data generated from an epidemic mechanistic model, and real influenza data from three different sources. The synthetic data is based on the mechanistic differential equation (SIRS model) with added noise. The Centers for Disease Control and Prevention reports the incidences of influenza-like illness (ILI) through its surveillance network. Web data of Twitter messages and Wikipedia article access logs are shown to be highly correlated with the ILI data; therefore, the relative real-time web data allows the robust prediction of influenza when the ILI data is not available due to delay in the surveillance network. Methods. Even though Kalman filters and particle filters are previously widely applied to engineering problems, the study of infectious diseases using filtering methods is a recent advancement. We first implement the filtering methods in combination with the mechanistic model to test the prediction ability using the synthetic data. The synthetic data allows a quantitative comparison based on the mean square error and the log likelihood for the different filtering methods. To study the real influenza trend, we implement the extended Kalman filter and particle filter using ILI data, Wikipedia and twitter signals in a recently developed empirical framework (Archetype framework). Results. In the experiments of the synthetic data, unscented Kalman filter yields the lowest mean square error and the highest log likelihood in comparison with extended Kalman filter, ensemble Kalman filter and particle filter. The mean square error from the unscented Kalman filter is 5% smaller than the ensemble Kalman filter. In forecasting the influenza trend, the real influenza observations are well within 80% confidence interval of one-week predictions using the Archetype framework. However the influenza peak prediction is lagged by 1 week than the observed influenza peak. Conclusions The filtering methods demonstrate fine performance in both the synthetic data and real influenza data. Filtering methods in the Archetype framework are simpler to implement, and yield good influenza predictions.